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PREFACE 


Since  the  late  1970s,  the  Logistics  Research  Division  of  the  Armstrong  Laboratory  has 
conducted  research  to  develop  and  evaluate  the  technology  for  an  Integrated  Maintenance 
Information  System  (BVUS).  IMIS  vvill  provide  maintenance  technicians  and  managers  with  the 
capability  to  access  all  the  technical  information  required  to  perform  their  jobs  via  a  single, 
integrated  system.  The  final  phase  of  the  IMIS  program  was  to  develop  a  demonstration  system 
incorporating  the  technologies  and  illustrating  the  capabilities  of  IMIS.  This  report  summarizes 
the  results  of  a  user  field  test  and  demonstration  conducted  to  evaluate  IMIS  and  demonstrate  its 
capabilities  to  operational  maintenance  personnel.  The  user  field  test  and  demonstration  was 
conducted  at  Luke  Air  Force  Base,  Arizona,  during  the  summer  of  1994. 

IMIS  is  the  product  of  the  creativity,  talents,  and  efforts  of  many  people,  including 
Armstrong  Laboratory  personnel  and  a  number  of  contractors.  The  IMIS  program  was  initiated 
and  directed  by  Mr  Robert  C.  Johnson,  AL/HR(jO.  The  program  was  accomplished  by  a 
dedicated  staff  of  scientists  and  engineers,  with  the  strong  support  of  three  Division  Chiefs; 

Mr  Bertram  W.  Cream,  Col  James  C.  Clark,  and  Col  Donald  C.  Tetmeyer. 

The  IMIS  project  was  conducted  under  Project  2950,  Work  Unit  2950-00-09.  The 
program  managers  were  Mr.  Richard  E.  Weimer  and  Major  Thomas  M.  Kruzick.  The  prime 
contractor  for  the  development  of  the  IMIS  Demonstration  System  was  GDE  Systems,  Inc.,  with 
its  subcontractors  Applied  Science  Associates,  Inc.,  Softech,  and  Systems  Control  Technology. 

In  addition,  technics  support  was  provided  to  the  Laboratory  by  NCI  Information  Systems,  the 
University  of  Dayton  Research  Institute,  Computer  Sciences  Corporation,  RJO  Enterprises,  and 
Robins-Gioia,  Inc.  Additional  support  was  received  from  the  Lockheed  Corporation,  under  the 
sponsorship  of  the  F- 16  System  Program  Office. 

The  user  demonstration  and  field  test  could  not  have  been  accomplished  without  the 
generous  support  of  the  host  unit,  the  3 10th  Fighter  Squadron.  The  efforts  of  Captains  Tribble 
and  Ramero,  CMSgt  Rios,  and  the  technicians  and  supervisors  under  their  command  were  a  major 
factor  in  the  success  of  the  field  test. 
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Integrated  Maintenance  Information  System: 

User  Field  Demonstration  and  Test 
Executive  Summary 

SYNOPSIS 

This  report  summarizes  the  results  of  the  final  phase  of  a  program  to  develop  and  evaluate 
the  concept  of  an  Integrated  Maintenance  Information  System  (IMIS).  IMIS  will  provide 
maintenance  technicians  and  managers  with  the  capability  to  access  all  the  technical  information 
required  to  perform  their  jobs  via  a  single,  integrated  system.  The  project  developed  and  field 
tested  the  technology  to  implement  the  concept.  The  technologies  were  then  implemented  in  a 
demonstration  system  for  evaluation.  Technological  developments  include  advances  in  the  areas 
of  Interactive  Electronic  Technical  Manuals  (lETMs);  interactive,  computer-generated 
diagnostics;  human/computer  interface;  special-purpose  portable  computers;  and  integration  of 
divergent  data  bases.  An  IMIS  demonstration  system,  applying  these  technologies,  was 
developed  and  evaluated. 

The  IMIS  program  was  established  to  evaluate  the  IMIS  concept  and  to  develop  and  test  the 
technology  required  for  an  operational  IMIS.  Major  activities  under  the  program  included: 

a.  comprehensive  analysis  of  maintenance  information  requirements  to  serve  as  the  baseline 
for  developing  IMIS; 

b.  research  to  develop  an  advanced,  interactive,  diagnostic-aiding  capability; 

c.  research  to  develop  effective  human/computer  interface  and  technical  information 
presentation  techniques  to  make  IMIS  easy  to  use  and  an  effective  means  of  communicating 
technical  information; 

d.  development  of  a  methodology  for  authoring  and  coding,  lETMs  (technical  orders 
[TOs])  in  an  efficient  and  cost-effective  manner  which  provides  flexibility  and  reduces 
redundancy; 

e.  development  of  demonstration  portable  maintenance  aids  (PMAs)  using  off-the-shelf 
components;  and 

f  development  of  an  IMIS  demonstration  system  which  incorporates  the  major  features  and 
functions  of  IMIS  for  use  in  evaluating  IMIS  technology  and  validating  requirements. 

The  IMIS  Demonstration  System  was  evaluated  in  a  three-phase  field  test  at  Luke  Air  Force 
Base,  Arizona,  in  the  spring  and  summer  of  1994.  This  report  summarizes  the  findings  of  the 
third  phase  of  the  evaluation,  a  test  to  evaluate  the  ability  of  the  system  to  support  on-aircrafl 
fault  isolation  and  repair  tasks.  In  the  test,  an  experiment  was  conducted  to  evaluate  the  impact 
of  IMIS  on  the  performance  of  maintenance  technicians.  In  the  experiment,  12  avionics 
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specialists  and  12  non-specialists  (airplane  general  [APG]  technicians)  performed  12  fault 
isolation  problems  on  three  F-16  subsystems:  the  fire  control  radar  (FCR),  heads-up  display 
(HUD),  and  Initertial  Navigation  System  (INS).  Half  the  problems  were  performed  using  the 
current  paper-based  technical  orders  TOs  and  part-ordering  and  documentation  procedures.  The 
APG  technicians  were  included  in  the  study  to  determine  if  the  use  of  IMIS  would  enable  non¬ 
specialist  technicians,  with  little  or  no  training  on  a  specific  aircraft  subsystem,  to  isolate  and 
repair  faults  in  that  system  at  least  as  effectively  as  specialists,  with  specific  experience  and 
training  on  the  system,  using  paper  TOs. 

Analysis  of  the  technicians  performance  on  the  12  test  problems  suggests  the  following 
benefits  from  IMIS. 

a.  The  rate  for  successful  problem  completion  was  improved  by  approximately 
22  percent  for  specialists  and  42  percent  for  APG  technicians. 

b.  The  frequency  of  serious  errors  (errors  which  could  lead  to  failure  to  correct  the 
problem)  were  reduced  by  58  percent  for  specialists  and  83  percent  for  APG  technicians. 

c.  Overall  problem  times  (troubleshoot,  order  parts,  repair,  and  document)  were  reduced  by 
approximately  17  percent  for  specialists  and  29  percent  for  APG  techmcians.  Analysis  indicates 
that  the  reductions  in  times  were  due  primarily  to  time  saved  by  the  automated  parts-ordering 
and  work  order  close-out  features  of  IMIS. 

d.  Overall  parts  consumption  for  the  12  problems  was  reduced  by  26  percent  for  specialists 
and  37  percent  for  APG  technicians.  In-depth  analysis  indicates  that  IMIS  diagnostics  are  much 
more  beneficial  for  the  more  complex  problems  (e.g.,  most  of  the  part  savings  were  from  the  INS 
system). 

e.  Part-ordering  times  were  reduced  by  94  percent  for  both  specialists  and  APG  techmcians. 

f  For  all  measures,  the  performance  of  the  APG  technicians  when  using  IMIS  was 
approximately  equal  to  the  performance  of  the  specialists  when  using  paper  TOs. 

A  key  finding  is  that  the  use  of  IMIS  had  the  effect  of  “leveling  the  playing  field”  for  the 
APG  technicians.  With  IMIS,  the  APG  technicians  were  able  to  perform  the  fault  isolation 
problems  as  effectively  as  specialists  using  IMIS  and  more  effectively  than  the  specialists  using 
paper  TOs.  This  finding  suggests  that  the  use  of  IMIS  will  enable  the  crew  chief  to  perform  a 
much  wider  range  of  tasks,  thus  reducing  reliance  on  specialists. 

The  responses  to  questionnaires  administered  to  the  technicians  after  completing  the  test 
problems  indicate  a  high  rate  of  acceptance  of  IMIS  and  a  desire  to  see  it  implemented  in  the  Ar 
Force.  IMIS  features  which  were  rated  highly  include  the  integration  of  technical  data, 
automated  parts-ordering  by  radio  frequency  (RF)  link,  automated  close-out,  and  automated 
diagnostic  advice. 
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BACKGROUND 


The  IMIS  project  was  an  advanced  development  demonstration  project  conducted  by  the 
Logistics  Research  Division  of  the  Armstrong  Laboratory.  The  project  developed  and  field 
tested  the  technology  to  provide  the  maintenance  technician  with  the  capability  to  access  all 
technical  information  (DETMS,  interactive  diagnostics  instructions,  work  orders,  supply 
availability  and  ordering,  historical  data,  training  material,  etc.)  required  to  maintain  aircraft  via  a 
single,  integrated  system,  regardless  of  the  source  of  that  information.  This  capability  will  help 
maintain  the  new  technology  found  in  advanced  weapon  systems  and  will  support  the  current 
trend  toward  fewer  maintenance  specialties.  In  the  future,  portable  computers  will  deliver  the 
electronic  data  at  the  flightline  work  site,  and  a  network  of  work  stations  will  interact  with 
existing  maintenance  computer  systems  in  shops  and  work  centers.  IMIS  is  designed  to  be 
totally  integrated  with  other  maintenance  information  systems  during  peacetime  at  main 
operating  bases,  but  the  basic  portable  diagnostic  and  TO  automation  is  fully  deployable  to 
remote  dispersed  locations  during  war  (Johnson,  1994). 

A  full  implementation  of  the  IMIS  concept  will  have  the  following  features  and  capabilities. 

•  lETMs  stored  and  presented  on  a  PMA. 

•  Interactive  diagnostic-aiding  via  a  PMA,  supported  by  an  interface  with  the  aircraft  data 
bus  to  operate  built-in-tests  and  down-load  system  performance  data  for  use  by  an 
IMIS  maintenance  diagnostic  algorithm. 

•  Interface  with  Core  Automated  Maintenance  System  (CAMS),  Standard  Base  Supply 
System  (SBSS),  and  other  information  systems  and  data  bases  which  support 
maintenance  operations. 

•  Training  materials  integrated  with  the  lETM  and  automated  diagnostics  data  base  to 
support  on-the-job  and  upgrade  training. 

•  Maintenance  data  collection,  via  the  CAMS  interface. 

•  Radio  communication  capability  between  PMAs  and  IMIS  local  area  network  (LAN) 
provides  digital  data  exchange  with  IMIS  LAN. 

•  IMIS  LAN  connects  IMIS  workstations,  RF  modems,  printers,  data  storage  devices, 
and  interfaces  with  external  data  bases. 
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•  IMIS  LAN  connects  IMIS  workstations,  RF  modems,  printers,  data  storage  devices, 
and  interfaces  with  external  data  bases. 

Fully  implemented,  IMIS  would  include  the  following  basic  hardware  and  software. 

•  PMAs,  four  to  six  pounds,  fully  ruggedized,  multiple  power  sources,  plug-in  memory 
modules,  screen  readable  under  all  conditions,  self-contained  batteries,  interface  to 
aircraft  maintenance  bus,  usable  with  chemical  and  cold  weather  gloves,  easy  to  operate 
without  training,  typing  skills  not  required. 

•  Stationary  Workstation  Computers,  to  support  maintenance  management  activities, 
maintain  the  IMIS  data  base,  and  interface  with  external  data  bases.  Located  in 
maintenance  management  and  support  offices. 

•  Mobil  Workstation  Computers  with  RF  capability,  mounted  in  maintenance 
supervisory  vehicles  for  use  in  managing  and  controlling  flightline  maintenance 
activities. 

•  Technical  data  presentation  software,  presents  lETM  data  and  diagnostic  procedures. 

•  Interactive  diagnostics  software,  generates  most  efficient  diagnostic  strategy  based 
upon  technical  data,  system  design  information,  component  failure  rates,  and  current 
system  status  (e.g.,  symptoms  and  test  results). 

•  IMIS  data  base,  maintains  data  required  to  support  IMIS  and  maintains  a  backup  copy 
of  the  relevant  CAMS  information  to  prevent  interruption  of  operations  if  CAMS  is 
down. 

•  External  Data  Base  Interface  software,  controls  interface  with  CAMS,  SBSS,  and  other 
external  data  bases. 

The  architecture  of  a  full-scale  implementation  of  IMIS  is  presented  in  Figure  1. 


IMIS  DEMONSTRATION  SYSTEM 

The  demonstration  system  incorporated  the  major  IMIS  functions,  including  enhanced 
debriefing,  presentation  of  TO  data,  automated  diagnostic-aiding  with  aircraft  data  bus  interface, 
RF  communications,  CAMS/SBSS  interface,  PMA,  and  advanced  human/computer  interface 
techniques.  The  IMIS  Demonstration  System  was  installed  in  the  facilities  of  the  3 10th  Fighter 
Squadron,  Luke  AFB.  The  IMIS  installation  at  Luke  AFB  consisted  of  a  network  of  four 
workstations  (located  in  the  debriefing  room.  Combat  Oriented  Support  Organization  [COSO], 
and  the  IMIS  field  office),  seven  PMAs,  and  two  laptop  computers  used  as  portable 
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Figure  1 

IMIS  Architecture 


workstations  in  the  production  superintendent  and  expediter  vehicles.  The  IMIS  workstations 
were  linked  by  a  fiber  optic  cable  to  form  the  IMIS  LAN.  The  LAN  was  connected  to  CAMS. 
See  Ward,  Weimer,  and  Kruzick  (1995)  for  a  detailed  description  of  the  IMIS  Demonstration 
System. 

The  IMIS  Demonstration  System  was  evaluated  in  a  three-phase  field  test.  These  phases 
are  described  in  the  following  paragraphs. 

Phase  I.  In  Phase  I,  the  IMIS  debriefing  function  was  evaluated.  The  results  of  this 
evaluation  indicate  that  IMIS  provides  an  effective  debriefing  capability.  Use  of  the  system 
resulted  in  faster  debriefs  for  Code  1  aircraft.  In  addition,  IMIS  has  the  capability  to  provide 
technicians  with  more  extensive  information  on  reported  system  failures  than  is  provided  by 
current  debriefing  procedures.*  Debriefers  who  used  the  system  found  it  effective  and  easy  to 
use. 


*  This  benefit  was  not  fully  evaluated  because  the  test  was  conducted  under  operational 
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Phase  n.  The  Phase  II  evaluation  was  an  "End-to-End"  Demonstration.  The  demonstration 
was  designed  to  evaluate  the  capability  of  IMIS  to  support  flightline  management  functions, 
especially  those  performed  by  the  production  superintendent  and  expediters.  To  avoid 
interfering  with  ongoing  maintenance  operations,  the  evaluation  was  conducted  in  a  classroom 
environment.  Expediters  and  production  superintendents  used  IMIS  to  perform  typical 
scenarios  incorporating  the  tasks  they  normally  perform  in  their  day-to-day  activities.  These 
activities  included  performing  actions  such  as  assigning  a  work  order  to  a  technician,  creating  and 
closing  work  orders,  approving  part  orders,  retrieving  aircraft  status  information,  changing 
aircraft  status,  and  sending  messages.  The  participants  were  asked  to  evaluate  the  system  as  an 
aid  for  performing  their  jobs;  the  evaluations  were  very  positive.  See  Ward,  Weimer,  and 
Kruzick  (1995)  for  additional  information  on  the  End-to-End  Demonstration. 

Phase  ni.  The  Phase  III  evaluation  (Field  Test)  was  designed  to  evaluate  the  capability  of 
IMIS  to  support  on-aircraft  maintenance,  especially  diagnostics.  The  results  of  this  test  are 
described  in  the  remainder  of  this  report. 


FIELD  TEST 

IMIS  is  expected  to  have  several  advantages  over  the  current  paper  TOs  for  supporting  the 
flightline  maintenance  processes.  These  advantages  include  reduced  time  to  accomplish 
troubleshooting  tasks,  reduced  spare  parts  consumption,  reduced  time  to  complete 
documentation,  and  more  accurate  maintenance  data  collection.  In  addition,  preliminary  studies 
have  indicated  that  the  use  of  IMIS  may  enable  technicians  without  extensive  training  on 
specific  systems  to  effectively  maintain  and  troubleshoot  those  systems.  For  example,  with  the 
use  of  EVnS,  it  has  been  suggested  that  an  APG  crew  chief  should  be  able  to  troubleshoot  as 
rapidly  and  accurately  as  avionics  specialists  are  able  to  troubleshoot  when  using  paper  TOs.  If 
this  proves  to  be  valid,  the  use  of  IMIS  would  have  major  implications  for  the  maintenance  force 
structure  because  it  could  somewhat  reduce  reliance  on  highly  trained  specialist  and  would  make 
greater  use  of  generalists  without  this  extensive  and  expensive  training.  The  IMIS  Constrained 
Test  was  conducted  to  test  the  above  assertions  and  to  provide  additional  data  for  use  in 
evaluating  the  overall  effectiveness  of  IMIS. 


conditions  and  the  current  paper-based  system  has  no  mechanism  available  to  provide  this 
information  to  the  technician.  In  a  full  implementation  of  IMIS,  this  information  would  be 
automatically  entered  into  the  IMIS  data  base,  loaded  onto  the  technician's  PMA,  and  used  by 
the  diagnostic  routine  to  isolate  the  fault. 
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Objectives 


The  objectives  of  the  Field  Test  were  to  demonstrate  that  use  of  IMIS  can  significantly 
improve  troubleshooting  performance  of  technicians  by  reducing  the  time  required  to  return  an 
aircraft  to  service,  spare  parts  consumption,  the  number  of  serious  errors  made  by  maintenance 
technicians,  the  time  to  order  parts,  and  the  time  to  complete  work  order  close-out  and 
documentation  procedures.  Additional  objectives  were  to  demonstrate  that  APG  technicians 
using  IMIS  are  able  to  perform  troubleshooting  tasks  on  F-16  avionics  subsystems  at  least  as 
effectively  as  specialists  using  paper  TOs  and  to  collect  data  for  use  in  evaluating  the  cost 
benefits  of  IMIS. 

To  accomplish  the  above  objectives,  it  was  necessary  to  compare  the  performance  of 
specialists  and  non-specialist  technicians  (APG  technicians)  using  IMIS  with  their  performance 
using  paper  TOs.  To  ensure  an  accurate  comparison,  it  was  essential  that  the  technicians’ 
performance  be  measured  under  comparable  conditions.  Thus,  it  was  necessary  to  measure  the 
performance  of  technicians  engaged  in  very  similar  tasks  under  very  similar  conditions  — 
preferably  doing  the  same  tasks  under  identical  conditions.  To  ensure  the  required  similarity  of 
tasks  and  data  collection  conditions,  an  experimental  approach  was  adopted. 


Experimental  Procedures 

An  experiment  was  conducted  in  which  avionics  specialists  and  APG  technicians  each 
performed  12  troubleshooting  tasks:  six  using  IMIS  and  six  using  paper  TOs.  The  order  of  task 
presentation  was  counterbalanced  so  that  half  the  technicians  performed  their  IMIS  tasks  first 
and  half  performed  their  paper-based  TO  tasks  first.  Twelve  troubleshooting  tasks  for  three 
F-16  subsystems  were  used  as  test  problems  for  the  study.  The  test-bed  systems  were  the 
F-16  FCR,  HUD,  and  INS.  The  tasks  were  selected  in  pairs  so  that  paired  tasks  were 
approximately  equal  in  difficulty  and  required  the  same  skills.  The  tasks  were  based  on  faults 
which  can  occur  in  the  test-bed  systems  and  are  considered  representative  of  the  troubleshooting 
tasks  normally  encountered  in  maintaining  these  test-bed  systems.  Breakout  boxes  were  used  to 
insert  the  faults  into  the  systems. 

The  technicians  were  closely  observed  as  they  completed  each  task.  A  data  collector  timed 
the  technicians  performance  and  recorded  data  on  problem  completion/failure,  errors  made,  and 
helps  given.^  To  successfully  complete  a  problem,  the  technician  was  required  to  verify  that 
the  reported  malfunction  existed  in  the  aircraft,  identify  the  faulty  component,  identify  the 


^  Helps  were  given  in  selected  situations,  when  requested  by  the  technician.  Helps  for  specialists 
primarily  were  limited  to  questions  on  the  use  of  IMIS.  A  more  liberal  policy  was  followed 
with  the  APG  technicians  because  they  were  unfamiliar  with  the  test-bed  systems.  APG 
technicians  often  required  help  in  locating  components  and  using  the  TO.  Frequently,  the  helps 
were  little  more  than  a  hint  (e.g.,  remember  what  you  were  taught  in  training). 
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required  repair,  order  any  required  part,  identify  the  checks  required  to  verify  that  the  repair 
returned  the  system  to  a  fully  operational  condition,  and  complete  work  order  close-out 
documentation.  Actual  repairs  were  not  made  in  order  to  reduce  the  risk  of  damaging  the 
aircraft.  Standard  times  were  used  to  account  for  the  times  required  to  remove  and  replace  parts 
and  for  the  final  system  health.  Also,  standard  times  were  used  to  account  for  the  time  to 
submit  part  orders  and  enter  close-out  data  into  CAMS  under  the  paper  TO  condition.  The  use 
of  standard  times  was  necessary  because  there  was  no  way  to  complete  these  tasks  under  the 
paper  TO  condition  without  interfering  with  squadron  operations. 

After  completing  the  assigned  problems  using  IMIS,  the  technicians  completed  an 
automated  questionnaire  designed  to  evaluate  various  features  or  qualities  of  IMIS  (e.g., 
readability  of  the  display).  After  they  had  completed  all  their  tests,  the  technicians  completed 
an  open-ended  questionnaire  which  asked  them  what  they  liked  about  IMIS,  what  they  did  not 
like,  and  how  IMIS  could  be  improved. 


Field  Test  Results 


Analysis  of  variance  was  used  to  evaluate  the  technicians'  performance  on  the  fault  isolation 
tasks.  The  findings  are  presented  below. 


Successful  Task  Completion 


The  technicians’  performance  was  evaluated  to  determine  if  they  had  satisfactorily 
completed  all  requirements  (as  defined  above).  The  percentage  of  problems  successfully 
completed  under  each  test  condition  (using  IMIS  or  paper  TOs)  was  computed.  These 
percentages  are  presented  in  Table  1. 


Table  1.  Percent  of  Problems  Successfully  Completed 
by  Avionics  Specialists  and  APG  Technicians 


TO 

IMIS 

Significant 

Avionics  Specialist 

81.9 

100.0 

Yes** 

APG  Technician 

69.4 

98.6 

Yes  *** 

Total 

75.7 

99.3 

Yes*** 

**  p<  .01 
***  p  <  .001 
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The  specialist  and  APG  technicians  successfully  completed  nearly  all  the  problems  when 
using  IMIS.  Only  one  problem  was  failed  when  using  IMIS  compared  to  26  (of  144)  problems 
failed  when  using  paper  TOs  as  the  source  of  technical  data.  Of  particular  interest  is  the  fact 
that  when  using  IMIS  the  APG  technicians  were  nearly  as  successful  in  completing  the  fault 
isolation  problems  as  were  the  avionics  specialists.  This  is  an  important  finding  because  it 
indicates  that,  with  IMIS,  crew  chiefs  could  perform  a  much  wider  variety  of  tasks,  reducing  the 
dependence  on  highly  trained  specialists. 

The  observed  differences  in  performance  with  IMIS  and  paper  TOs  are  statistically 
significant  for  both  specialists  and  APG  technicians.  Also,  the  difference  in  the  observed 
success  rate  for  technicians  using  IMIS  versus  avionics  specialists  using  paper  TOs  is 
statistically  significant. 

The  success  rate  for  both  specialists  and  APG  technicians  was  much  lower  when  the  TO 
was  the  source  of  technical  data.  Close  examination  of  the  data  reveals  that  most  of  the  failures 
with  the  TO  were  due  to  a  failure  to  complete  all  the  required  system  health  checks.  The 
difference  in  performance  can  be  explained  by  the  fact  that  one  or  more  built-in-tests  or 
operational  checks  are  required  to  verify  that  the  system  has  been  returned  to  operational  status. 
System  health  tests  and  checkout  requirements  are  presented  in  the  follow-on  maintenance 
requirements  section  of  the  TO.  The  manner  in  which  the  follow-on  maintenance  requirements 
are  presented  in  the  TOs  for  some  systems  makes  it  easy  to  overlook  required  checks.  As  a 
result,  several  technicians  failed  to  complete  all  the  required  checks  and  failed  the  problem.  With 
IMIS,  it  is  impossible  to  overlook  the  required  checks.  When  a  technician  completes  a  task, 
IMIS  automatically  presents  the  instructions  for  the  follow-on  task.  The  technician  must  follow 
the  instructions  or  consciously  choose  not  to  do  the  task. 


Parts  Used 

The  mean  number  of  parts  used  by  each  technician  to  complete  the  six  problems  under  each 
condition  is  shown  in  Table  2.  The  specialists  required  an  average  of  8.67  parts  to  complete  the 
six  problems  using  the  TO,  compared  to  6.42  parts  when  using  IMIS.^  The  APG  technicians 


^Three  of  each  set  of  six  problems  required  replacement  of  a  part  to  correct  the  fault.  The 
remaining  faults  were  caused  by  wiring  and  required  no  parts.  Thus,  rectification  of  the  problems 
required  three  parts  per  subject,  per  condition.  Any  parts  used  in  excess  of  three  were  "good" 
parts  either  replaced  by  the  diagnostic  strategy  or  because  of  an  error  by  the  technician.  The 
diagnostic  strategy  employed  in  the  TOs  required  replacement  of  five  good  parts  for  the  six 
problems.  The  diagnostic  strategy  generated  by  IMIS  required  the  replacement  of  two  good 
parts  for  the  six  problems.  The  F-16  TOs  often  direct  the  replacement  of  a  component  to 
determine  if  it  is  good  or  bad.  This  normally  occurs  when  there  is  no  test  available  to  determine  if 
it  is  good  or  bad,  or  because  the  troubleshooting  procedure  does  not  take  advantage  of  an  available 
test. 
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required  8.30  parts  for  the  problems  when  using  the  TO,  compared  to  5.30  parts  when  using 
DVDS.  Again,  it  should  be  noted  that,  on  this  measure,  the  APG  technicians  were  as  proficient 
as  the  avionics  specialists. 


Table  2.  Mean  Number  of  Parts  Used  by  Each  Technician 
for  Six  Problems  Under  Each  Condition _ 


TO 

IMIS 

Significant 

Avionics  Specialists 

8.67 

6.42 

Yes*** 

APG  Technicians 

8.30 

5.30 

Yes*** 

Total 

8.48 

5.84 

Yes*** 

***  p  <  .0001 


Detailed  analysis  of  parts  usage  revealed  that  the  great  majority  of  the  part  savings  for 
EMIS  were  from  one  subsystem:  the  INS.  This  appears  to  be  due  to  the  differences  in  the 
complexity  of  the  troubleshooting  tasks  for  the  system.  INS  troubleshooting  procedures  are 
much  more  complex  than  procedures  for  the  FCR  and  HUD.  Additional  analyses  are  being 
performed  to  evaluate  the  observed  differences. 


Task  Performance  Times 

The  mean  times  for  technicians  to  perform  their  assigned  tasks  using  either  IMIS  or  paper 
TOs  were  computed.  The  means  are  presented  in  Table  3. 


Table  3.  Mean  Problem  Performance  Times 
_ (in  Minutes)  for  Each  Problem _ 


TO 

IMIS 

Significant 

Avionics  Specialists 

149.29 

123.64 

Yes** 

APG  Technicians 

175.82 

124.04 

Yes*** 

Total 

161.46 

123.83 

Yes*** 

**  p  <  .01 
***  p  <  .001 


Both  the  avionics  specialists  and  APG  technicians  required  significantly  longer  to  complete 
the  fault  isolation  problems  when  using  the  TO.  Use  of  IMIS  reduced  the  problem  performance 
times  of  the  specialists  by  approximately  17  percent  and  the  times  of  the  APG  technicians  by 
approximately  29  percent.  The  performance  times  of  the  specialists  and  APG  technicians  were 
essentially  the  same,  indicating  that  the  APG  technicians  using  IMIS  were  able  to  perform  the 
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job  as  efTiciently  as  the  avionics  specialists  (and  more  efficiently  than  the  avionics  specialists 
using  their  current  methods). 

A  more  detailed  analysis  of  the  performance  times  was  conducted  to  identify  which 
elements  of  IMIS  contribute  the  most  to  the  observed  reductions  in  performance  times.  The 
analysis  indicated  that  nearly  all  the  observed  differences  were  due  three  factors: 

a.  the  reduction  in  the  number  of  good  parts  replaced, 

b.  the  reduction  in  the  time  required  to  order  parts  when  IMIS  is  used,  and 

c.  the  reduction  in  the  time  required  to  complete  work  order  close-out  documentation. 


Part-Ordering  Time 

By  reducing  the  number  of  good  parts  unnecessarily  replaced,  IMIS  reduces  the  time 
required  to  isolate  and  repair  a  system  fault.  Time  savings  were  realized  by  eliminating 
unnecessary  tasks,  such  as  removing  a  good  part,  replacing  it  with  a  new  part,  and  performing 
system  health  checks  to  determine  that  the  new  part  did  not  fix  the  problem. 

A  large  percentage  of  the  observed  total  time  difference  between  the  IMIS  and  paper  TO 
conditions  was  due  to  the  difference  in  the  way  parts  are  ordered  under  the  two  systems.  When 
using  the  paper  TO,  the  technician  must  go  to  COSO,  look  up  the  part  number,  obtain 
authorization  to  order  the  part,  and  submit  the  part  order  to  the  COSO  clerk  who  must  input 
the  order  into  the  SBSS.  Thus,  ordering  parts  is  a  time-consuming  process  (a  conservative 
estimate  of  15  minutes  per  part  ordered  was  used  for  this  study).  In  contrast,  when  using  IMIS, 
technicians  are  asked  if  they  want  to  order  the  part.  If  they  answer  “yes,”  IMIS  automatically 
submits  the  order  by  RF  link  to  the  Production  Superintendent  for  approval.  IMIS  then 
submits  the  approved  order  to  the  SBSS.  While  IMIS  is  processing  the  part  order,  the 
technician  is  free  to  remove  the  defective  part  or  perform  other  maintenance  activities.  Thus,  at 
least  15  minutes  are  saved  per  part  order. 

The  difference  in  mean  part-ordering  times  using  IMIS  versus  the  current  parts-ordering 
procedure  are  illustrated  in  Table  4.  As  may  be  observed  from  the  table,  the  time  savings 
resulting  from  the  use  of  IMIS  are  dramatic.  The  observed  differences  are  statistically 
significant,  well  beyond  the  0.001  level  of  confidence. 


Table  4.  Mean  Time  (in  Minutes)  to 


Complete  Each  Part  Order 


TO 

IMIS 

Significant 

Avionics  Specialists 

19.42 

1.16 

APG  Technicians 

25.28 

1.47 

Yes*** 

Total 

22.35 

1.32 

Yes*** 

***  p  <  .001 


Close-Out  Time 

The  third  primary  source  of  time  savings  is  from  the  use  of  IMIS's  work  order  close-out 
and  RF  functions  to  enter  close-out  information  into  CAMS.  With  a  full  implementation  of  the 
IMIS  concept,  IMIS  will  automatically  record  all  information  required  to  complete  the  work 
order  close-out  process.  When  the  job  is  completed,  the  technician  will  instruct  the  system  to 
assemble  the  work  order  close-out  information;  the  information  will  be  presented  to  the 
technician  for  verification  and  correction,  if  needed.  After  verification  by  the  technician,  the 
information  will  be  sent  by  RF  to  CAMS  to  complete  the  work  order  close-out  process.  Under 
the  current  procedures,  the  technician  must  make  notes  on  actions  taken,  parts  used,  part 
numbers,  and  so  forth  during  the  fault  isolation  and  repair  process.  The  technician  must  then  go 
to  the  maintenance  office,  find  a  CAMS  terminal,  and  enter  the  information  from  the  notes  taken 
(or  from  memory). 

The  IMIS  demonstration  system  did  not  fully  implement  the  IMIS  concept  for  work  order 
close-out.  The  system  did  not  automatically  record  all  the  required  information.  The  system 
presented  a  form  (similar  to  Air  Force  Technical  Order  [AFTO]  Form  349)  with  some  blocks 
filled  in  and  others  to  be  completed.  The  technician  filled  in  the  blanks  by  selecting  from  lists  of 
options.  When  the  form  was  completed,  the  information  was  transmitted  by  RF  to  the  IMIS 
workstation  for  forwarding  to  CAMS.  As  indicated  earlier,  it  was  not  possible  to  enter  the 
closeout  information  into  CAMS  for  the  TO-based  condition.  To  provide  an  estimate  of  the 
times  to  close  out  a  work  order  with  the  current  procedures,  the  technician  completed  a  paper 
form  with  the  required  information.  The  time  required  to  complete  the  form,  plus  a  standard 
time  (10  minutes)  was  used  as  an  estimate  of  the  time  it  would  have  taken  to  close  the  work 
order  using  the  current  procedures.  The  mean  observed  times  are  presented  in  Table  5.  The 
differences  in  observed  close-out  times  for  both  the  specialists  and  APG  technicians  were 
statistically  significant  at  the  0.001  level  of  confidence.  In  addition,  the  times  for  the  APG 
technicians  using  IMIS  were  significantly  shorter  than  the  times  for  the  specialists  using  the 
current  CAMS-based  procedures  (p  <  .001).  In  a  full  implementation  of  IMIS,  the  required 
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information  would  automatically  be  collected  and  used  to  complete  the  data-reporting 
requirement.  The  technician  would  not  have  to  add  information,  only  verify  that  the 
information  is  correct.  Thus,  the  time  for  the  IMIS  condition  would  be  near  zero. 

Table  5.  Mean  Times  (in  Minutes)  to 


Close  Out  Each  Problem 


TO 

IMIS 

Significant 

Avionics  Specialists 
APG  Technicians 

14.67 

17.31 

8.17 

8.82 

Yes*** 

Yes*** 

Total 

15.98 

8.49 

Yes*** 

***  p  <  .001 


Errors 

It  was  anticipated  that  the  use  of  IMIS  would  reduce  the  number  of  errors  made  by  the 
technicians.  As  shown  in  Table  6,  this  expectation  was  realized.  The  use  of  IMIS  resulted  in  a 
dramatic  reduction  in  serious  maintenance  errors  (errors  which  could  cause  the  fault  not  to  be 
identified  or  cause  the  unnecessary  replacement  of  a  good  part).  The  use  of  IMIS  resulted  in  a 
56-percent  reduction  in  major  errors  made  by  the  specialists  and  an  82-percent  reduction  in 
major  errors  by  the  APG  technicians.  These  observed  differences  were  statistically  significant 
at  the  0.001  level  of  confidence.  In  addition,  the  APG  technicians  using  IMIS  made  significantly 
fewer  major  errors  than  did  the  specialist  technicians  using  the  paper  TOs  (p  <  .001). 


Table  6.  Mean  Number  of  Major  Errors  per  Problem 


TO 

IMIS 

Significant 

Avionics  Specialists 

.69 

.29 

No 

APG  Technicians 

1.06 

.18 

Yes*** 

Total 

.87 

.23 

Yes*** 

***p  <  .001 


¥ 
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IMIS  Characteristics  Questionnaire  Results 


Examination  of  the  responses  to  the  IMIS  Characteristics  Questionnaire  reveals  that  the 
various  features  of  IMIS  were  generally  rated  very  positively.  The  most  highly  rated  features 
include  using  IMIS  to  order  parts  and  close  work  orders,  the  ease  of  completing  forms,  the  ease 
of  using  the  IMIS  fault  isolation  procedures,  and  the  ease  of  navigating  through  the  TO.  Items 
which  received  negative  ratings  included  the  response  time  of  the  IMIS  PMA,  the  weight  of  the 
PMA,  and  the  PMA  keyboard. 


Exit  Questionnaire  Comments 

The  written  comments  provided  by  the  technicians  at  the  end  of  the  test  indicate  that  the 
technicians  have  a  very  positive  view  of  IMIS.  The  exit  questionnaire  statements  clearly 
demonstrate  that  the  technicians  like  the  concept  of  IMIS  and  believe  it  has  great  potential.  This 
observation  is  based  upon  statements  such  as: 

Great  project  -  quite  a  bit  better  than  having  to  use  TOs.  I  hope  I  see  it  in  use  somewhere, 
sometime  before  I  get  out  of  the  service  or  even  in  the  civilian  world. 

IMIS  definitely  needs  to  be  an  integral  part  of  the  AF.  The  AF  will  benefit  greatly  from  the 
implementation  of  IMIS. 


Very  good  unit.  If  it  were  not  for  the  waiting  on  the  computer,  it  would  be  great. 
A  complete  listing  of  the  exit  questionnaire  comments  is  presented  in  Appendix  A. 


DISCUSSION  AND  CONCLUSIONS 

The  IMIS  field  test  successfully  demonstrated  the  feasibility  of  the  IMIS  concept  and  the 
potential  benefits  of  developing  the  system  for  operational  Air  Force  use.  The  Debriefing  and 
End-to-End  tests  demonstrated  the  ability  of  IMIS  to  enhance  the  debriefing  process  and  to 
improve  the  efficiency  of  maintenance  management  functions  (Ward  et  al,  1995).  The  IMIS 
Field  Test  provides  strong  evidence  of  IMIS’s  capability  to  enhance  the  performance  of 
maintenance  technicians  performing  on-aircraft  maintenance.  It  also  provides  the  basis  for 
determining  the  potential  cost  savings  which  will  accrue  from  the  implementation  of  IMIS  in  the 
Air  Force.  The  primary  findings  of  the  Field  Test  and  their  implications  are  discussed  briefly 
below. 


IMIS  Concept  Validation 

The  Field  Test  clearly  demonstrated  that  the  use  of  IMIS  for  on-aircraft  maintenance  will 
improve  technician  performance.  When  using  IMIS,  technicians  were  able  to  complete  fault 
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isolation  and  repair  problems  with  greater  accuracy,  in  a  shorter  time,  and  with  fewer  errors.  In 
addition,  fewer  parts  were  used  in  the  process.  The  improved  performance  will  significantly 
reduce  the  time  required  to  return  an  aircraft  to  operationally  capable  status,  provide  for  more 
effective  utilization  of  available  personnel,  and  reduce  expenditures  for  procuring,  repairing,  and 
stocking  aircraft  replacement  components. 

The  Field  Test  and  End-to-End  demonstration  clearly  showed  the  advantages  of  the  IMIS 
RF  link  capability.  These  advantages  were  most  clearly  demonstrated  in  the  Field  Test,  where  it 
was  found  that  using  the  RF  for  ordering  parts  and  transmitting  work  order  close-out 
information  was  the  major  contributor  to  faster  job  completion.  The  End-to-End  Demonstration 
showed  the  advantages  of  the  IMIS  RF  capability  in  supporting  flightline  maintenance  managers 
in  a  variety  of  ways,  including  providing  up-to-the-minute  aircraft  and  maintenance  status 
information  and  offering  the  capability  to  make  personnel  assignments,  open  and  close  work 
orders,  and  communicate  with  other  maintenance  managers. 

The  Field  Test  demonstrated  that  IMIS  is  easy  to  use  and  is  preferred  by  technicians.  The 
technicians  had  very  little  trouble  using  IMIS.  After  receiving  a  short  training  session,  both  the 
specialist  and  APG  technicians  were  able  to  perform  the  test  tasks  with  minimal  difficulty.  In 
contrast,  the  APG  technicians  (and  some  specialists)  experienced  significant  difficulties  in  using 
the  paper  TOs. 

The  IMIS  user  interface  provides  an  effective  means  of  retrieving  the  information  that 
technicians  require  to  perform  on-aircraft  maintenance.  The  test  identified  a  number  of 
modifications  to  the  user  interface  which  would  make  it  more  effective.  These  include 
presenting  more  than  one  step  at  a  time  so  the  technician  can  look  ahead  and  see  the  next  step, 
providing  a  browsing  capability,  and  providing  better  locator  information.  The  IMIS  user 
interface  provides  a  good  starting  point  for  the  development  of  the  user  interface  for  an 
operational  IMIS.  Designers  of  an  operational  IMIS  would  benefit  greatly  from  a  careful  study 
of  the  Demonstration  System  interface. 


IMIS  and  the  Non-Specialist  Technician 

One  objective  of  the  study  was  to  demonstrate  that,  when  using  IMIS,  APG  technicians 
could  troubleshoot  complex  systems  as  effectively  as  avionics  specialists  could  troubleshoot  the 
problems  when  using  TOs.  This  objective  was  exceeded.  The  APG  technicians  were 
significantly  more  successful  in  performing  the  problems  with  IMIS  than  were  the  avionics 
specialists  using  the  TO. 

The  above  finding  has  significant  implications  for  how  the  Air  Force  trains  and  assigns 
maintenance  personnel.  It  suggests  that,  with  IMIS,  the  crew  chief  can  be  assigned  a  much 
broader  range  of  tasks  than  is  possible  with  the  current  paper-based  TO  system.  However,  the 
finding  applies  only  when  the  crew  chief  is  using  an  IMIS-like  system  with  high-quality 
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technical  data  which  directly  leads  to  the  isolation  of  most  faults.  The  crew  chief  will  not  have 
the  background  necessary  to  handle  those  occasional  problems  not  covered  by  the  IMIS 
technical  data;  the  highly  trained  specialist  will  still  be  required  to  resolve  these  problems. 
Furthermore,  for  the  crew  chief  to  be  assigned  to  the  broader  range  of  tasks,  he  or  she  will 
require  additional  training.  This  training  would  include  an  orientation  to  the  systems  to  be 
maintained  (including  component  location,  operation,  and  basic  maintenance  procedures)  and 
training  on  any  required  test  equipment  (e.g.,  use  of  multimeter).  The  additional  training  should 
require  the  addition  of  only  a  few  days  to  the  current  APG  training. 


Lessons  Learned 

Many  lessons  were  learned  in  conducting  the  IMIS  program.  Some  of  the  more  important 
lessons  are  described  below.  Additional  lessons  learned  and  discussions  are  provided  in  the 
basic  complete  test  report  (Thomas,  1995). 

a.  The  user  interface  is  critical.  The  IMIS  user  interface  proved  to  be  quite  effective  and 
was  a  major  factor  in  the  positive  response  of  the  technicians  to  the  system. 

b.  The  quality  of  the  technical  data  used  by  IMIS  is  critical.  The  data  used  for  the  Field 
Test  was  converted  directly  from  the  existing  paper  TOs.  As  a  result,  many  of  the  problems  in 
the  TOs  were  carried  over  to  the  IMIS  data  base  (e.g.,  inadequate  illustrations,  insufficient 
diagnostic  tests,  etc.).  Consequently,  the  full  potential  of  IMIS  to  improve  performance  was 
not  demonstrated  in  this  test.  The  addition  of  a  few  more  tests,  a  few  more  illustrations,  and 
clarification  of  some  confusing  text  could  have  reduced  the  number  of  parts  consumed  and  errors 
made. 

c.  Adequate  test  points  are  essential  for  effective  diagnostics.  To  achieve  the  full  potential 
of  the  IMIS  diagnostics,  the  aircraft  system  must  be  designed  with  sufficient  test  points  to 
allow  the  isolation  of  a  fault  to  a  single  component.  Adequate  test  points  were  not  available  for 
the  test-bed  systems,  limiting  the  effectiveness  of  the  IMIS  diagnostics.  Had  adequate  test 
points  been  available  and  included  in  the  technical  data,  the  number  of  parts  used  under  the 
IMIS  condition  would  have  been  significantly  reduced. 

d.  IMIS  diagnostics  may  be  more  appropriate  for  some  systems  than  others.  The  IMIS 
diagnostics  appear  to  be  most  beneficial  for  complex  systems  with  lengthy,  complex 
troubleshooting  procedures. 
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Concluding  Comments 


The  MIS  Field  Test  clearly  demonstrates  that  the  MIS  has  a  great  potential  for  improving 
the  effectiveness  and  efficiency  of  Air  Force  maintenance  operations.  The  next  step  is  to  build 
an  MIS  for  operational  use.  This  project  has  provided  a  firm  basis  for  the  development  of  the 
operational  MIS.  The  Air  Force  has  directed  that  the  MIS  capabilities  be  implemented  in  a 
new  information  system  currently  under  development.  This  system,  the  Integrated 
Maintenance  Data  System  (MDS),  will  be  a  full  implementation  of  the  MIS  concept.  The 
MIS  Demonstration  System  has  been  transitioned  to  the  MDS  Program  Office  for  further 
testing  and  evaluation. 
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ACRONYMS 


AFB 

Air  Force  Base 

AFTO 

Air  Force  Technical  Order 

AIP 

Aircraft  Interface  Panel 

APG 

Airplane  General 

CAMS 

Core  Automated  Maintenance  System 

CEMS 

Centralized  Maintenance  Management  System 

COSO 

Combat  Oriented  Support  Organization 

FCR 

Fire  Control  Radar 

HUD 

Heads-Up  Display 

lETM 

Integrated  Electronic  Technical  Manuals 

IMDS 

Integrated  Maintenance  Data  System 

IMIS 

Integrated  Maintenance  Information  System 

INS 

Inertial  Navigation  System 

JCALS 

Joint  Continuous  Acquisition  Lifecycle  Support 

LAN 

Local  Area  Network 

MIW 

Maintenance  Information  Workstation 

PMA 

Portable  Maintenance  Aid 

RF 

Radio  Frequency 

SBSS 

Standard  Base  Supply  System 

TO 

Technical  Order 
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APPENDIX  A 


EXIT  QUESTIONNAIRE  COMMENTS 
Question:  What  did  you  like  about  IMIS  as  an  aid  to  doing  your  job? 

Comments: 

The  speed  with  which  you  and  IMIS  can  do  a  Job,  the  graphics  were  good  for  aiding  with 
wire  troubleshooting. 

The  whole  concept  is  pretty  good.  Go  to  aircraft,  diagnose,  order  part,  not  have  to  carry 
paper. 

The  speed  in  diagnosing  and  accomplishing  a  task. 

When  the  PMA  was  connected  to  the  1553  Bus,  it  was  very  quick  and  accurate. 

Parts  ordering,  debriefing  and  job  close-out  were  much  simpler  than  using  CAMS. 

Information  easily  accessible,  accurate,  easy  to  understand. 

It  cut  down  on  the  mental  frustration  of  my  job  and  the  amount  of  tech  data  that  I  would 
normally  have  had  to  go  through  to  do  my  job.  Also,  the  part  ordering  feature  cut 
down  part  ordering  time  considerably. 

It  cut  the  time  down  to  nothing. 

You  had  one  unit  instead  of  many  TOs.  It  gives  you  only  necessary  information,  less 
confusion. 

It  makes  sure  everything  is  done  in  order. 

The  integration  of  all  TOs  and  the  ease  of  ordering  parts. 

The  troubleshooting  fault  tree  and  the  job  guides.  They  replicated  the  paper  ones  very 
well. 
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It  was  very  straightforward.  It  took  everything  associated  with  the  system  into 

consideration,  and  gave  the  choice  to  the  technician  as  to  the  action  he  feels  to  be 
best.  It  made  ordering  parts  very  simple,  actually  it  does  it  for  you.  It  saves  time  on 
paper  work.  I  feel  with  IMIS  that  anybody,  regardless  of  experience,  can  work  on 
the  aircraft. 

The  ease  of  closing  work  orders  and  the  ordering  of  parts.  The  troubleshooting  was  also 
helpful. 

Simplified  troubleshooting. 

Easier  than  carrying  a  library  of  TOs. 

Saved  effort  in  fumbling  through  books  when  shooting  wires. 

Convenient  way  to  order  parts. 

Integration  with  aircraft  systems  (BIT  checks),  faster  than  the  aircraft  (GAC). 

Made  troubleshooting  easier. 

It  made  troubleshooting  much  easier  and  eliminated  tedious  job  of  checking  out  tech  data 
and  equipment  from  support.  I  also  feel  that  IMIS  will  decrease  the  down  time  of 
the  aircraft. 

IMIS  was  very  helpful  in  all  aspects  of  completing  a  job  from  starting  the  work  order,  to 
verifying  a  fault,  and  to  rectifying  the  fault.  The  most  impressive  aspect  of  IMIS  is 
parts  order! 

If  the  system  is  used  on  the  flightline,  it  could  speed  up  different  jobs  and  take  away  a  lot 
of  other  frustration  with  dealing  with  the  different  TOs. 

The  auto  BIT  feature  was  extremely  helpful,  and  the  recommended  best  action  was  a  good 
feature  as  well. 

It  made  unknown  tasks  easy  to  understand;  parts  ordering  and  job  closings  were  fast. 

I  found  it  very  helpful.  With  me  being  fairly  new  to  my  job,  I  hadn't  used  hardly  any  of 
the  TO  material,  as  far  as  FI  are  concerned.  From  just  learning  to  use  both,  after 
some  basic  TO  (paper)  training  I  found  IMIS  much  more  easier. 

It  takes  up  the  repetitious  workload. 
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Parts  ordering,  CAMS  interface,  not  having  to  carry  a  lot  of  books. 
Makes  troubleshooting  very  easy. 

It  works  very  well  as  aid. 


Question:  What  did  you  dislike  about  IMIS. 

Comments: 

There  is  nothing  I  can  think  of  that  would  make  me  dislike  this  system. 

Transition  times  between  screens. 

Some  text  is  misleading.  For  example,  hook  up  of  the  PMA  to  A/C  could  be  clearer.  INS 
task  seemed  to  require  more  thought. 

I  personally  read  over  the  job  guide  before  I  do  a  job.  With  IMIS,  you  can't  really  do  that. 

Periodic  delays  between  screens. 

It  still  had  lock-ups  and  that  you  did  not  have  the  option  to  go  back  if  you  happened  to  hit 
the  wrong  button  during  operational  checks. 

Typical  prototype  bugs. 

The  user  interface.  The  hourglass  should  consistently  go  to  a  place  on  the  screen  where  it 
is  visible.  As,  the  PMA  is  transitioning,  it  is  quite  confusing.  There  is  no  cue  that 
the  machine  is  busy.  It  actually  prints  "Press  FI  for  OK,"  when  you  can't.  Also, 
having  three  methods  to  select  an  option  is  not  a  good  idea.  Use  FI,  F8,  and  Select 
consistently,  not  FI  for  this  F8  for  that  and  Select  for  something  else.  Use  either  the 
thumb  knob  or  arrow  keys. 

The  fact  that  steps  were  given  only  one  step  at  a  time  in  the  job  guide  portion.  It  can  slow 
down  a  job. 

It  was  a  little  heavy.  A  keyboard  would  have  been  convenient.  When  troubleshooting 
wires,  it  could  be  more  specific  about  the  whereabouts  of  some  of  the  components, 
i.e.,  "station  88  disconnect...."  A  little  slow  at  times. 

The  speed  of  the  software.  No  link  between  part  number  and  serial  number  removed  (with 
work  order  close-out). 
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IMIS  is  slow. 


Slow  to  process  information  at  times. 

Keys  sometimes  hard  to  press;  desired  result  doesn't  occur. 

At  times  was  a  little  slow. 

In  the  short  time  I  used  the  IMIS,  I  can't  really  say  that  I  disliked  anything  about  the 
system. 

The  pros  definitely  outweigh  the  cons.  However,  IMIS  seems  to  sometimes  get  into  slow 
modes,  i.e.,  changing  from  screen  to  screen,  giving  completed  task  information. 

I  did  not  like  the  reliability  factor.  If  you  are  in  the  middle  of  a  job  and  it  goes  down,  it 
tends  to  be  frustrating. 

The  time  it  took  to  process  during  RF  operations,  the  amount  of  breakdowns,  and  a  few 
minor  misleading  or  irritating  display  errors. 

The  screen  is  difficult  to  see  unless  you  are  looking  at  just  the  right  angle.  Sometimes 
when  it  is  thinking,  it  doesn't  let  you  know  what  it  is  doing  and  you  are  apt  to  get 
frustrated  with  it  when  you  try  to  do  something  and  it  won’t  respond. 

About  the  only  thing  I  didn't  like  about  IMIS  is  the  thumb  control  for  the  cursor. 

Nothing. 

Speed,  poor  pictures,  one  step  at  a  time.  A  page  down  or  up  to  proceed  would  be  nice. 

It's  too  hard  to  get  back  to  a  step  you  miss.  It  takes  too  long  to  change  screens  at  certain 
times. 

How  slow  it  is  in  processing  information!  Having  to  skip  from  FI  to  the  select  key. 


m;  What  Changes  would  you  make  to  improve  the  system? 


Comments: 


Less  redundancy,  e.g.,  safe  for  maintenance  question  being  asked  over  and  over.  Make  safe 
for  maintenance  an  option  to  get  to  if  you  want  to. 
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Allow  the  maintainer  the  option  to  look  over  the  job  before  doing  a  job,  so  there  are  no 
surprises. 

Speed  of  processing  improvements. 

Try  and  add  some  more  prompts  to  the  screens  on  choices.  Breakdown  pictures  of  Lures 
caald  be  a  little  more  clear.  Color  screens  would  be  great. 

More  detailed  pictures,  faster  response  time. 

Fix  user  interface  (see  previous  user  interface  comments). 

Speed  up  the  reaction  time.  It  was  slow  at  times. 

Speed  up  the  PMAs. 

Continuity  in  using  the  system,  i.e.,  no  shifting  between  FI  and  SELECT  keys. 

Give  better  instructions  for  some  of  the  more  obscure  items,  for  example,  disconnects  and 
transformer  assemblies. 

If  you  remove  a  part  it  should  update  the  work  order.  The  speed  and  everything  else  was 
fine. 

Process  faster. 

Add  schematic/wiring  diagram  type  diagrams.  This  is  a  must.  From  time  to  time  we  see  a 
problem  that  "can't  happen"  according  to  the  FI.  In  other  words,  there  is  nothing 
listed  for  "Pilot  Detectable"  or  "Maintenance  Detectable"  faults.  Thus  there  is  no 
fault  tree  to  follow.  Since  the  avionics  systems  of  the  F-16  are  so  integrated,  a 
component  not  in  the  "suspected  bad"  system  can  cause  a  malfunction.  The  only 
way,  in  most  cases,  to  troubleshoot  these  kind  of  discrepancies  is  to  break  out  the 
schematics  and  "chase"  the  signal  flow. 

Response  time. 

Other  than  a  few  minor  computer  glitches,  the  system  worked  well.  It  would  be  nice  if  the 
PMA  could  operate  longer  on  a  charged  battery. 

Make  the  system  convert  from  page  to  page  faster  and  fix  the  RF  problems. 

Improve  reliability  and  response  time. 
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Increase  the  amount  of  back  light,  drop  a  glare  shield  on  it  (maybe  with  solar  panels),  put  a 
zoom  feature  on  the  pictures. 

Try  to  speed  it  up  on  some  things,  make  the  keyboard  a  little  more  user  friendly,  unless 
you  are  really  careful,  the  screws  can  become  a  FOD  hazard  in  the  cockpit. 

I  think  that  maybe  a  separate  detachable  mouse  or  maybe  a  track  ball  would  be  a  good 
improvement. 

Quicker  response  times  to  the  commands. 

Speed. 

Use  a  process  or  unit  to  speed  it  up. 

Make  the  system  memory  faster.  Sometimes  it  takes  awhile  between  choices. 

Speed  up  some  of  the  reaction  times.  It  was  slow  at  some  times. 


Question:  Other  comments? 

Comments: 

Good/excellent  idea. 

IMIS  would  be  an  outstanding  system  to  supplement  existing  tech  data.  However,  its 
ability  to  present  the  "big  picture"  of  a  problem  appears  to  be  limited  and  it  does  not 
explain  why  something  is  happening. 

Thank  you  for  the  training  on  the  jet.  The  hands-on  troubleshooting  was  very  helpful. 
People  "in  charge"  were  friendly  and  informative  as  to  what  was  going  on  with  the 
program. 

It  is  difficult  to  form  a  real  opinion  of  the  system  at  this  time,  as  a  fair  Judgment  can't 
really  be  made  until  the  system  is  truly  "in  the  field."  But,  I  like  what  I  see  so  far, 
and  have  every  bit  of  confidence  that  your  team  will  perfect  this  system  and  put  it 
into  use.  It  has  been  a  pleasure  to  work  with  you. 


Overall  a  good  system. 


IMIS  definitely  needs  to  be  an  integral  part  of  the  AF.  The  AF  will  benefit  greatly  from 
the  implementation  of  IMIS. 

I  think  this  program  is  a  good  one  and  has  been  very  educational  for  me. 

Great  project  -  quite  a  bit  better  than  having  to  use  TOs.  I  hope  I  see  it  in  use  somewhere, 
sometime  before  I  get  out  of  the  service  or  even  in  the  civilian  world. 

I  really  hope  this  system  takes  effect  soon. 

Very  good  unit.  If  it  were  not  for  the  waiting  on  the  computer,  it  would  be  great. 

I  like  the  idea  and  the  concept!  I  hope  to  see  it  in  the  future  on  all  A/C  systems. 
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